Skip to content

Giraffe server engine - #4879

Open
parsaeskandar wants to merge 58 commits into
vgteam:masterfrom
parsaeskandar:giraffe-server-engine
Open

parsaeskandar wants to merge 58 commits into
vgteam:masterfrom
parsaeskandar:giraffe-server-engine

Conversation

@parsaeskandar

@parsaeskandar parsaeskandar commented Apr 20, 2026 •

Copy link
Copy Markdown
Contributor

Changelog Entry

To be copied to the draft changelog by merger:

  • New vg giraffe-server subcommand that loads the Giraffe indexes once and maps queries streamed on stdin, with optional surjection onto any haplotype path.
  • vg giraffe preset blat, which reports up to 20 near-best alignments with MAPQ independent of query length.

Description

This PR adds a server-oriented in-process Giraffe mapping API and extends giraffe-server to emit structured graph-path payloads that middleware can forward to coordinate-translation code.

parsaeskandar and others added 3 commits April 16, 2026 09:03
Refactor Giraffe usage into a reusable in-process engine that loads indexes once and maps batched reads repeatedly, and add a giraffe-server subcommand with framed output/flush control for stable middleware request-response behavior.
@parsaeskandar
parsaeskandar requested a review from adamnovak April 20, 2026 17:53

@adamnovak adamnovak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@parsaeskandar You wanted me to check this for overly tight coupling to what Giraffe does. I don't think it should have to change too much when Giraffe changes. We won't be able to avoid touching it if we e.g. add a new thing MininizerMapper needs to have loaded in order to work, but I'm not sure there's a great way to avoid that.

But I think you might not connect to the Giraffe machinery quite enough, because there's no way here to e.g. set the mapping parameters that need to change between HiFi and R10 reads (and there's no code to set them to good values for pretending to be BLAT). I'm not sure we're even turning on the chain-based mapping. The Giraffe preset system is all tied in with its option parsing system, which makes using it from here difficult, but maybe it needs to be broken out somehow?

There also seems to be a profusion of little classes defined mostly in relation to each other, like HaplotypeSurjectionResult and GiraffeEnginePaths, which could do with more explanation of what they really represent or mean for someone who does not, to begin with, care about a HaplotypeSurjector or a GiraffeEngine.

Comment thread src/subcommand/giraffe_server_main.cpp Resolved
Comment thread src/subcommand/giraffe_server_main.cpp Resolved
Comment thread src/subcommand/giraffe_server_main.cpp Resolved
Comment thread src/subcommand/giraffe_server_main.cpp Resolved
Comment thread src/subcommand/giraffe_server_main.cpp Resolved
Comment thread src/haplotype_assignment.hpp Resolved
Comment thread src/giraffe_engine.hpp Resolved
Comment thread src/giraffe_engine.hpp Resolved
Comment thread src/giraffe_engine.hpp Resolved
Comment thread src/giraffe_engine.hpp Resolved
@parsaeskandar parsaeskandar linked an issue Sep 21, 2026 that may be closed by this pull request
parsaeskandar and others added 4 commits October 3, 2026 12:32
scripts/lint.py (formerly check_options.py) now also requires every
subcommand in doc/vgmanmd.desc.md and doc/vgmanmd.py. That check arrived with
the merge of master, so make test has failed on this branch since then.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolves the doc/vgmanmd.py conflict (upstream deprecated genotype and viz;
keep giraffe-server in the list) and follows upstream's rename of
min_chains -> target_alignment_attempts in the engine's hifi and blat
presets, without which the engine no longer compiles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@adamnovak adamnovak added the mapping Theme/team for mapping to pangenomes label Nov 2, 2026
…nd the server names the haplotypes when it writes the tags
@parsaeskandar

Copy link
Copy Markdown
Contributor Author

Thanks @adamnovak. I went through all of the code by hand this time. Here is a summary of what changed:

  • vg giraffe and vg giraffe-server both fill giraffe option parser now. Both call set_up_giraffe_mapper() so all presetts, options, rec mode, ... are the same code in both.
  • The PANGENOME_* environment variables are all gone.
  • In haplotype_assignment, the assigner now answers two separate questions about an alignment’s path.
    • Which haplotypes follow the path? In HaplotypeSegment
    • Where do those haplotypes leave the snarl around the path? snarl_boundaries().
  • in the vg giraffe-server I put the rule for requests and errors at the top of the giraffe_server_main.cpp. Now every request get exactly one frame (a result, or an error) and the frames are the only output format. The nonsense threadpool is not gone and I am using OpenMP threading.
  • Changed Engine to know nothing about GAF now and the server now writes GAF lines and tags.
  • I added a wiki that should be enough to for using vg giraffe-server: https://github.com/vgteam/vg/wiki/Giraffe-server-protocol
  • The timings, and FLUSH_NOW are gone.
  • Also the client can see the relevant errors on what went wrong, but one error or bad input won't break the whole process.

@parsaeskandar

Copy link
Copy Markdown
Contributor Author

I also hand copied this into the giraffe-server-engine branch so the tests can run.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

mapping Theme/team for mapping to pangenomes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Genome Browser Intergration

2 participants